Improving Fairness of Automated Chest X-ray Diagnosis by Contrastive Learning (preprint)

Purpose: Limited studies exploring concrete methods or approaches to tackle and enhance model fairness in the radiology domain. Our proposed AI model utilizes supervised contrastive learning to minimize bias in CXR diagnosis. Materials and Methods: In this retrospective study, we evaluated our proposed method on two datasets: the Medical Imaging and Data Resource Center (MIDRC) dataset with 77,887 CXR images from 27,796 patients collected as of April 20, 2023 for COVID-19 diagnosis, and the NIH Chest X-ray (NIH-CXR) dataset with 112,120 CXR images from 30,805 patients collected between 1992 and 2015. In the NIH-CXR dataset, thoracic abnormalities include atelectasis, cardiomegaly, effusion, infiltration, mass, nodule, pneumonia, pneumothorax, consolidation, edema, emphysema, fibrosis, pleural thickening, or hernia. Our proposed method utilizes supervised contrastive learning with carefully selected positive and negative samples to generate fair image embeddings, which are fine-tuned for subsequent tasks to reduce bias in chest X-ray (CXR) diagnosis. We evaluated the methods using the marginal AUC difference ($\delta$ mAUC). Results: The proposed model showed a significant decrease in bias across all subgroups when compared to the baseline models, as evidenced by a paired T-test (p<0.0001). The $\delta$ mAUC obtained by our method were 0.0116 (95\% CI, 0.0110-0.0123), 0.2102 (95% CI, 0.2087-0.2118), and 0.1000 (95\% CI, 0.0988-0.1011) for sex, race, and age on MIDRC, and 0.0090 (95\% CI, 0.0082-0.0097) for sex and 0.0512 (95% CI, 0.0512-0.0532) for age on NIH-CXR, respectively. Conclusion: Employing supervised contrastive learning can mitigate bias in CXR diagnosis, addressing concerns of fairness and reliability in deep learning-based diagnostic methods.

Disparity in the Evolving COVID-19 Collaboration Network (preprint)

The COVID 19 pandemic has paused many ongoing research projects and unified researchers' attention to focus on COVID 19 related issues. Our project traces 712294 scientists' publications related to COVID 19 for two years, from January 2020 to December 2021, to detect the dynamic evolution patterns of the COVID 19 collaboration network over time. By studying the collaboration network of COVID 19 scientists, we observe how a new scientific community has been built in preparation for a sudden shock. The number of newcomers grows incrementally, and the connectivity of the collaboration network shifts from loose to tight promptly. Even though every scientist has an equal opportunity to start a study, collaboration disparity still exists. Following the scale-free distribution, only a few top authors are highly connected with other authors. These top authors are more likely to attract newcomers and work with each other. As the collaboration network evolves, the increase rate in the probability of attracting newcomers for authors with higher degrees increases, whereas the increase rates in the likelihood of forming new links among authors with higher degrees decreases. This highlights the interesting trend that the COVID pandemic alters the research collaboration trends that star scientists are starting to collaborate more with newcomers but less with existing collaborators, which, in a certain way, reduces the collaboration disparity.

Analyzing Impact of Socio-Economic Factors on COVID-19 Mortality Prediction Using SHAP Value (preprint)

This paper applies multiple machine learning (ML) algorithms to a dataset of de-identified COVID-19 patients provided by the COVID-19 Research Database. The dataset consists of 20,878 COVID-positive patients, among which 9,177 patients died in the year 2020. This paper aims to understand and interpret the association of socio-economic characteristics of patients with their mortality instead of maximizing prediction accuracy. According to our analysis, a patients households annual and disposable income, age, education, and employment status significantly impacts a machine learning models prediction. We also observe several individual patient data, which gives us insight into how the feature values impact the prediction for that data point. This paper analyzes the global and local interpretation of machine learning models on socio-economic data of COVID patients.

Using Explainable AI to Cross-Validate Socio-economic Disparities Among Covid-19 Patient Mortality (preprint)

This paper applies eXplainable Artificial Intelligence (XAI) methods to investigate the socioeconomic disparities in COVID patient mortality. An Extreme Gradient Boosting (XGBoost) prediction model is built based on a de-identified Austin area hospital dataset to predict the mortality of COVID-19 patients. We apply two XAI methods, Shapley Additive exPlanations (SHAP) and Locally Interpretable Model Agnostic Explanations (LIME), to compare the global and local interpretation of feature importance. This paper demonstrates the advantages of using XAI which shows the feature importance and decisive capability. Furthermore, we use the XAI methods to cross-validate their interpretations for individual patients. The XAI models reveal that Medicare financial class, older age, and gender have high impact on the mortality prediction. We find that LIME local interpretation does not show significant differences in feature importance comparing to SHAP, which suggests pattern confirmation. This paper demonstrates the importance of XAI methods in cross-validation of feature attributions.

Durable T cell responses contrast with faster antibody waning in BNT162b2-vaccinated elderly at 6 month (preprint)

Efficient COVID-19 vaccines have been developed in record time. Here, we present findings from a comprehensive and integrated analysis of multiple compartments of the memory immune response in 312 individuals vaccinated with the BNT162b2 mRNA vaccine. Two vaccine doses induced high antibody and T cell responses in most individuals. However, antibody recognition of the Spike protein of delta variant was less efficient than that of the Wuhan strain. Age stratified analyses identified a group of low antibody responders where individuals ≥ 60 years were overrepresented. Waning of the antibody and cellular responses was observed in 30% of the vaccinees after six months. However, age did not influence the waning of these responses. Taken together, while individuals ≥ 60 years old took longer to acquire vaccine-induced immunity, they develop more sustained acquired immunity at six months post-vaccination. However, the higher proportion of older individuals in the group of antibody low responders and the lower antibody reactivity the Delta variant call for a booster immunization to increase immune responses and protection.

Rapid measurement of SARS-CoV-2 spike T cells in whole blood from vaccinated and naturally infected individuals.

Defining the correlates of protection necessary to manage the COVID-19 pandemic requires the analysis of both antibody and T cell parameters, but the complexity of traditional tests limits virus-specific T cell measurements. We tested the sensitivity and performance of a simple and rapid SARS-CoV-2 spike protein-specific T cell test based on the stimulation of whole blood with peptides covering the SARS-CoV-2 spike protein, followed by cytokine (IFN-γ, IL-2) measurement in different cohorts including BNT162b2-vaccinated individuals (n = 112), convalescent asymptomatic and symptomatic COVID-19 patients (n = 130), and SARS-CoV-1-convalescent individuals (n = 12). The sensitivity of this rapid test is comparable to that of traditional methods of T cell analysis (ELISPOT, activation-induced marker). Using this test, we observed a similar mean magnitude of T cell responses between the vaccinees and SARS-CoV-2 convalescents 3 months after vaccination or virus priming. However, a wide heterogeneity of the magnitude of spike-specific T cell responses characterized the individual responses, irrespective of the time of analysis. The magnitude of these spike-specific T cell responses cannot be predicted from the neutralizing antibody levels. Hence, both humoral and cellular spike-specific immunity should be tested after vaccination to define the correlates of protection necessary to evaluate current vaccine strategies.

Rapid determination of the wide dynamic range of SARS-CoV-2 Spike T cell responses in whole blood of vaccinated and naturally infected (preprint)

Background: Antibodies and T cells cooperate to control virus infections. The definition of the correlates of protection necessary to manage the COVID-19 pandemic, require both immune parameters but the complexity of traditional tests limits virus-specific T cell measurements. Methods: We test the sensitivity and performance of a simple and rapid SARS-CoV-2 Spike-specific T cell test based on stimulation of whole blood with peptides covering the SARS-CoV-2 Spike protein followed by cytokine (IFN-{gamma}, IL-2) measurement in different cohorts including BNT162b2 vaccinated (n=112; 201 samples), convalescent asymptomatic (n=62; 62 samples) and symptomatic (n=68; 115 samples) COVID-19 patients and SARS-CoV-1 convalescent individuals (n=12; 12 samples). Results: The sensitivity of the rapid cytokine whole blood test equates traditional methods of T cell analysis (ELISPOT, Activation Induced Markers). Utilizing this test we observed that Spike-specific T cells in vaccinated preferentially target the S2 region of Spike and that their mean magnitude is similar between them and SARS-CoV-2 convalescents at 3 months after vaccine or virus priming respectively. However, a wide heterogeneity of Spike-specific T cell magnitude characterizes the individual responses irrespective of the time of analysis. No correlation between neutralizing antibody levels and Spike-specific T cell magnitude were found. Conclusions: Rapid measurement of cytokine production in whole blood after peptide activation revealed a wide dynamic range of Spike-specific T cell response after vaccination that cannot be predicted from neutralizing antibody quantities. Both Spike-specific humoral and cellular immunity should be tested after vaccination to define the correlates of protection necessary to evaluate current vaccine strategies.

Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data (preprint)

Electronic Health Record (EHR) data has been of tremendous utility in Artificial Intelligence (AI) for healthcare such as predicting future clinical events. These tasks, however, often come with many challenges when using classical machine learning models due to a myriad of factors including class imbalance and data heterogeneity (i.e., the complex intra-class variances). To address some of these research gaps, this paper leverages the exciting contrastive learning framework and proposes a novel contrastive regularized clinical classification model. The contrastive loss is found to substantially augment EHR-based prediction: it effectively characterizes the similar/dissimilar patterns (by its "push-and-pull" form), meanwhile mitigating the highly skewed class distribution by learning more balanced feature spaces (as also echoed by recent findings). In particular, when naively exporting the contrastive learning to the EHR data, one hurdle is in generating positive samples, since EHR data is not as amendable to data augmentation as image data. To this end, we have introduced two unique positive sampling strategies specifically tailored for EHR data: a feature-based positive sampling that exploits the feature space neighborhood structure to reinforce the feature learning; and an attribute-based positive sampling that incorporates pre-generated patient similarity metrics to define the sample proximity. Both sampling approaches are designed with an awareness of unique high intra-class variance in EHR data. Our overall framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data with a total of 5,712 patients admitted to a large, urban health system. Specifically, our method reaches a high AUROC prediction score of 0.959, which outperforms other baselines and alternatives: cross-entropy(0.873) and focal loss(0.931).

Contrastive Learning Improves Critical Event Prediction in COVID-19 Patients (preprint)

Machine Learning (ML) models typically require large-scale, balanced training data to be robust, generalizable, and effective in the context of healthcare. This has been a major issue for developing ML models for the coronavirus-disease 2019 (COVID-19) pandemic where data is highly imbalanced, particularly within electronic health records (EHR) research. Conventional approaches in ML use cross-entropy loss (CEL) that often suffers from poor margin classification. For the first time, we show that contrastive loss (CL) improves the performance of CEL especially for imbalanced EHR data and the related COVID-19 analyses. This study has been approved by the Institutional Review Board at the Icahn School of Medicine at Mount Sinai. We use EHR data from five hospitals within the Mount Sinai Health System (MSHS) to predict mortality, intubation, and intensive care unit (ICU) transfer in hospitalized COVID-19 patients over 24 and 48 hour time windows. We train two sequential architectures (RNN and RETAIN) using two loss functions (CEL and CL). Models are tested on full sample data set which contain all available data and restricted data set to emulate higher class imbalance.CL models consistently outperform CEL models with the restricted data set on these tasks with differences ranging from 0.04 to 0.15 for AUPRC and 0.05 to 0.1 for AUROC. For the restricted sample, only the CL model maintains proper clustering and is able to identify important features, such as pulse oximetry. CL outperforms CEL in instances of severe class imbalance, on three EHR outcomes with respect to three performance metrics: predictive power, clustering, and feature importance. We believe that the developed CL framework can be expanded and used for EHR ML work in general.

Pandemics are catalysts of scientific novelty: Evidence from COVID-19 (preprint)

Scientific novelty drives the efforts to invent new vaccines and solutions during the pandemic. First-time collaboration and international collaboration are two pivotal channels to expand teams' search activities for a broader scope of resources required to address the global challenge, which might facilitate the generation of novel ideas. Our analysis of 98,981 coronavirus papers suggests that scientific novelty measured by the BioBERT model that is pre-trained on 29 million PubMed articles, and first-time collaboration increased after the outbreak of COVID-19, and international collaboration witnessed a sudden decrease. During COVID-19, papers with more first-time collaboration were found to be more novel and international collaboration did not hamper novelty as it had done in the normal periods. The findings suggest the necessity of reaching out for distant resources and the importance of maintaining a collaborative scientific community beyond nationalism during a pandemic.

Are we there yet? Analyzing scientific research related to COVID-19 drug repurposing (preprint)

Drug repurposing may be a pivotal means of fulfilling urgent needs for treatment of the novel coronavirus disease 2019 (COVID-19), but current studies on drug repurposing for COVID-19 seem to show a lack of consensus in their drug candidate focus. Using bibliometric methods in a non-expert perspective, in a review of 34 published articles on the COVID-19 and drug-repurposing, we investigated obvious and less obvious points of consensus on drug candidates. To establish these two types of consensus, we first implemented document clustering. Within a set of five clustered papers, we established an obvious consensus, relying solely on the occurrence of entities by using term frequency and inverse document frequency and a comparison of mentioned drugs, finding that remdesivir and chloroquine were discussed with a certain degree of agreement. For the less obvious consensus, we created a drug entity co-occurrence network to establish low-high centrality combinations to probe the crucial drugs found in article clustering that are not plainly apparent through the mere counting of the occurrence of drug entities occurrences. Lopinavir emerged as having possibly potent effects in spite of underuse, while the mainstream of studies focus more on drugs such as chloroquine that enjoy explicit consent. Using an entitymetrics perspective, we expect that our research will support investigations of drug repurposing, expediting the process of establishing treatment for COVID-19.

Coronavirus Knowledge Graph: A Case Study (preprint)

The emergence of the novel COVID-19 pandemic has had a significant impact on global healthcare and the economy over the past few months. The virus's rapid widespread has led to a proliferation in biomedical research addressing the pandemic and its related topics. One of the essential Knowledge Discovery tools that could help the biomedical research community understand and eventually find a cure for COVID-19 are Knowledge Graphs. The CORD-19 dataset is a collection of publicly available full-text research articles that have been recently published on COVID-19 and coronavirus topics. Here, we use several Machine Learning, Deep Learning, and Knowledge Graph construction and mining techniques to formalize and extract insights from the PubMed dataset and the CORD-19 dataset to identify COVID-19 related experts and bio-entities. Besides, we suggest possible techniques to predict related diseases, drug candidates, gene, gene mutations, and related compounds as part of a systematic effort to apply Knowledge Discovery methods to help biomedical researchers tackle the pandemic.

Analysis of misinformation during the COVID-19 outbreak in China: cultural, social and political entanglements (preprint)

COVID-19 resulted in an infodemic, which could erode public trust, impede virus containment, and outlive the pandemic itself. The evolving and fragmented media landscape is a key driver of the spread of misinformation. Using misinformation identified by the fact-checking platform by Tencent and posts on Weibo, our results showed that the evolution of misinformation follows an issue-attention cycle, pertaining to topics such as city lockdown, cures, and preventions, and school reopening. Sources of authority weigh in on these topics, but their influence is complicated by peoples' pre-existing beliefs and cultural practices. Finally, social media has a complicated relationship with established or legacy media systems. Sometimes they reinforce each other, but in general, social media may have a topic cycle of its own making. Our findings shed light on the distinct characteristics of misinformation during the COVID-19 and offer insights into combating misinformation in China and across the world at large.